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Abstract. This paper presents the Anisotropic selection scheme for cellular Gen- 
etic Algorithms (cGA). This new scheme allows to enhance diversity and to 
control the selective pressure which are two important issues in Genetic Algori- 
thms, especially when trying to solve difficult optimization problems. Varying 
the anisotropic degree of selection allows swapping from a cellular to an island 
model of parallel genetic algorithm. Measures of performances and diversity have 
been performed on one well-known problem: the Quadratic Assignment Problem 
which is known to be difficult to optimize. Experiences show that, tuning the 
anisotropic degree, we can find the accurate trade-off between cGA and island 
models to optimize performances of parallel evolutionary algorithms. This trade- 
off can be interpreted as the suitable degree of migration among subpopulations 
in a parallel Genetic Algorithm. 



Introduction 



In the context of cellular genetic algorithm (cGA), this paper proposes the Anisotropic 
selection as a new selection scheme which accurately allows to adjust the selective pres- 
sure and to control the exploration/exploitation ratio. This new class of evolutionary al- 
gorithms is supervised in a continuous way by an unique real parameter a in the range 
[-1..1]. The work described in this paper is an attempt to provide a unified model of par- 
allel genetic algorithms (pGA) from fine grain massively parallel GA (cGA) to coarse 
grain parallel model (island GA). As extreme cases, there are the cGA that assumes one 
individual resides at each cell, and at the opposite, a pGA where distinct subpopulations 
execute a standard GA; between them we find models of pGA where migration allows to 
exchange to some extend genetic information between subpopulations. Thus the search 
dynamics of our family of pGA can vary from a diffusion to a migration process. To 
illustrate our approach we used one well-known problem: the Quadratic Assignment 
Problem (QAP). We study the performances of our class of parallel evolutionary algo- 
rithms on this problem and we show that there is a threshold for parameter a according 
to the average performances. Section 1 gives a description of the cGA and the island 
models. Section 2 introduces the anisotropic parallel Genetic Algorithms (apGA) and 
the anisotropic selection scheme. Section 3 is a presentation of the test problem: the 
QAP, and gives the performances of the apGA on the QAP. Finally, a study on popula- 
tion genotypic diversity is made in section 4. 



1 Background 



This section introduces the concepts of Cellular and Island Models of parallel genetic 
algorithms. 

1.1 Cellular Genetic Algorithms 

The Cellular Genetic Algorithms are a subclass of Evolutionnary Algorithms in which 
the population is generally embedded on a two dimensional toroidal grid. In this kind of 
algorithms, exploration and population diversity are enhanced thanks to the existence 
of small overlapped neighborhoods (9j. An individual of the population is placed on 
each cell of the grid and represents a solution of the problem to solve. An evolution- 
nary process runs simultaneously on each cell of the grid, selecting parents from the 
neighborhood of the cells and applying operators for recombination, mutations and re- 
placement for further generations. Such a kind of algorithms is especially well suited 
for complex problems |5 1. One of the interests of cGA is to slow down the convergence 
of the population among a single individual. Complex problems often have many local 
optima, so if the best individual spreads too fast in the population it will improve the 
chances to reach a local optimum of the search space. Slowing down the convergence 
speed can be done by slowing down the selective pressure on the population. 

1.2 Island Model of pGA 

Cellular genetic algorithms and Island Model genetic algorithms are two kinds of Par- 
allel genetic algorithms. The first one is a fine grain massively parallel implementation 
that assumes one individual resides at each cell. The second one, using distinct subpop- 
ulations, is a coarse grain parallel model; Each subpopulation executes as a standard 
genetic algorithm, and occasionally the subpopulations would exchange a few strings: 
migration allows subpopulations to share genetical material [4|. Many topologies can 
be defined to connect the islands. In the basic island model, migration can occur be- 
tween any subpopulations, whereas in the stepping stone model islands are disposed on 
a ring and migration is restricted to neighboring islands. 

2 Anisotropic Parallel Genetic Algorithms 

This section presents the anisotropic parallel Genetic Algorithms, which is a family of 
parallel genetic algorithms based on cellular GA in which anisotropic selection is used. 

2.1 Definition 

The Anisotropic selection is a selection method in which the neighbors of a cell may 
have different probabilities to be selected. The Von Neumann neighborhood of a cell C 
is defined as the sphere of radius 1 centered at C in manhattan distance. The Anisotropic 
selection assigns different probabilities to be selected to the cells of the Von Neumann 
neighborhood according to their position. The probability to choose the center cell C 



remains fixed at |. Let us call p ns the probability of choosing the cells North (N) or 
South (S) and p ew the probability of choosing the cells East (E) or West (W). Let 
a E [—1; 1] be the control parameter that will determine the probabilities p ns and p ew . 
This parameter will be called the anisotropic degree. The probabilities p ns and p ew can 
be described as: 



Pew = ^ ^ ~ ") 

Thus, when a = —1 we have p ew = 1 — p c and p ns = 0. When a — 0, we have 
Pns = Pew and when a = 1, we have p n;s = 1 — p c and p eu , = 0. 
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Fig. 1. Von Neumann neighborhood with probabilities to choose each neighbor 



Figure 1 shows a Von Neumann Neighborhood with the probabilities to select each 
cell as a function of a. 

The Anisotropic Selection operator works as follows. For each cell it selects k indi- 
viduals in its neighborhood (fee [1; 5]). The k individuals participate to a tournament 
and the winner replaces the old individual if it has a better fitness or with probability 0.5 
if the fitnesses are equal. When a = 0, the anisotropic selection is equivalent to a stan- 
dard tournament selection and when a = 1 or a = —1 the anisotropy is maximal and 
we have an uni-dimensional neighborhood with three neighbors only. In the following, 
considering the grid symmetry we will consider a e [0; 1] only: when a is in the range 
[-1;0] making a rotation of 90° of the grid is equivalent to considering a in the range 
[0;1]. When the anisotropic degree is null, there is no anisotropy in selection method, 
the apGA corresponds to the standard cellular GA. When the anisotropic degree is max- 
imal, selection is computed between individuals in the same column only, the apGA is 
then an island model where each subpopulation is a column of the grid structured as a 
ring of cells with no interactions between subpopulations. When the anisotropic degree 
is set between low and maximum value, according to selection, a number of individuals 
can be copied from one subpopulation (i.e. column) to the adjacent columns. Thus the 
anisotropic degree allows to define a family of parallel GA from a cellular model to an 
island model. 



In standard island model, the migration rate is defined as the number of individuals 
which are swap between subpopulations and migration intervals is the frequency of mi- 
gration. In apGA, the migration process is structured by the grid. Only one parameter 
(the anisotropic degree) is needed to tune the migration policy. There is a difference 
between migration in a standard island model and migration in an apGA. In an apGA it 
can only happen (when the anisotropic degree allows it) between nearest neighbors in 
adjacent columns. Migration in that latter case is diffusion as it happens in the standard 
cGA model, except that the direction is controllable. In the following sections, we study 
the influence of this parameter on selection pressure, performances and population di- 
versity. 

2.2 Takeover times and apGAs 

The selective pressure is related to the population diversity in cellular genetic algo- 
rithms. One would like to slow down the selective pressure when trying to solve multi- 
modal problems in order to prevent the algorithm from converging too fast upon a local 
optimum. On the opposite side, when there is no danger of converging upon a local opti- 
mum, one would like to increase the selective pressure in order to obtain a good solution 
as fast as possible. A common analytical approach to measure the selective pressure is 
the computation of the takeover time O iflOll . It is the number of generations needed 
for the best individual to conquer the whole grid when the only active operator is the se- 
lection 0. Figure|2] shows the influence of the anisotropic degree on the takeover time. 
This figure represents the average takeover times observed on 1000 runs on a 32 x 32 
grid for different anisotropic degrees. It shows that the selective pressure is decreasing 
while increasing anisotropy. These results confirm that the anisotropic selection gives 
to the algorithm the ability to control accurately the selective pressure. They are fairly 
consistent with our expectation that selection intensity decreases when the anisotropic 
degree increases. However, the correlation between takeover and anisotropy is not lin- 
ear; it fast increases after the value a — 0.9. 

3 Test problem 

This section presents tests on one well-known instance of the Quadratic Assignment 
Problem which is known to be difficult to optimize. Our aim is to study the dynamics 
of the apGA for different tunings, and not to obtain better performances than other 
optimization techniques. Still, the apGA is implicitely compared to a cellular genetic 
algorithm when the anisotropic degree is null (a = 0). 

3.1 The Quadratic Assignment Problem 

We experimented the family of apGAs on a Quadratic Assignment Problem (QAP): 
Nug30. Our purpose here is not to obtain better results with respect to other optimization 
methods, but rather to observe the behavior of apGAs. Especially we go in the search 
of a threshold for the anisotropic degree. 
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Fig. 2. Average of the takeover time as a function of the anisotropic degree a. 

The QAP is an important problem in theory and practice as well. It was introduced 
by Koopmans and Beckmann in 1957 and is a model for many practical problems [6 |. 
The QAP can be described as the problem of assigning a set of facilities to a set of loca- 
tions with given distances between the locations and given flows between the facilities. 
The goal is to place the facilities on locations in such a way that the sum of the products 
between flows and distances is minimal. 

Given n facilities and n locations, two n x n matrices D = [dij] and F = [fkl\ where 
dij is the distance between locations i and j and the flow between facilities k and I, 
the objective function is: 



where p(i) gives the location of facility i in the current permutation p. 
Nugent, Vollman and Ruml proposed a set of problem instances of different sizes noted 
for their difficulty |2|. The instances they proposed are known to have multiple local 
optima, so they are difficult for a genetic algorithm. We experiment our algorithm on 
the 30 variables instance called Nug30. 

3.2 Setup 

We use a population of 400 individuals placed on a square grid (20 x 20). Each individ- 
ual represents a permutation of {1,2,..., 30}. We need a special crossover that preserves 
the permutations: 

- Select two individuals pi andj>2 as genitors. 

- Choose a random position i. 

- Find j and k so that p\ (%) — pi{j) and piii) =Pt(k). 




- exchange positions i and j from pi and positions i and k from p2- 

- repeat n/3 times this procedure where n is the length of an individual. 

This crossover is an extended version of the UPMX crossover proposed in [ 7 ] . The 
mutation operator consist in randomly selecting two positions from the individual and 
exchanging those positions. The crossover rate is 1 and we do a mutation per individual. 
We perform 500 runs for each anisotropic degree. Each run stops after 1500 generations. 



3.3 Experimental results 




Fig. 3. Average costs as a function of a for the QAP. 



Figure [3] shows the average performance of the algorithm towards a on the QAP: 
for each value of a we average the best solution of each run. The purpose here is to min- 
imize the fitness function values. The performances are growing with a and then fall 
down as a is getting closer to its limit value. The best average performance is achieved 
for a — 0.86. This threshold probably corresponds to a good exploration/exploitation 
trade-off: the algorithm favors propagation of good solutions in the vertical direction 
with few interactions on the left or the right sides. This kind of dynamics is well adapted 
to this multi-modal problem as we can reach local optima on each columns of the grid 
and then migrate them horizontally to find new solutions. The worst average perfor- 
mance is observed for a = when the apGA is a cellular GA. a = 0.86 corresponds 
to the optimal trade-off between cellular and island models for this problem, with the 
best migration rate between subpopulations. In our model, the migration rate is not the 
number of individuals which are swap between subpopulations, but the probability for 
the selection operator to choose two individuals from separate columns: two individuals 
from separate subpopulations would then share information. We can tell that there is an 



optimal migration rate that is induced by the value of the anisotropic degree a. Per- 
formances would probably improve if the migration rate did not stay static during the 
search process. As in (TJ, we can define some criteria to self-adjust the anisotropic de- 
gree along generations. 

4 Diversity in apGAs 

To understand better why we observe influence of the anisotropic parameter on perform- 
ances, we felt it is important to measure genetic diversity during runs. We studied 
changes in diversity during runs according to the whole grid, the rows and the columns. 

This section presents measures on population diversity in an apGA for the QAP. We 
conducted experiences on the average population diversity observed along generations 
on 100 independent runs for each anisotropic degree. We made three measures on the 
population diversity. First, we computed the global population diversity gD: 

gD = ( fl^ )2 ^ d(x riC1 ,x r2C2 ) 

V H n,r2ci,C2 

where d{x\,X2) is the distance between individuals x\ and X2- The distance used is 
inspired from the Hamming distance: It is the number of locations that differs between 
two individuals divided by their length n. 

Then, we made measures on diversity inside subpopulations (vertical diversity) and 
diversity between subpopulations (horizontal diversity). The vertical (resp. horizontal) 
diversity is the sum of the average distance between all individuals in the same column 
(resp. row) divided by the number of columns (resp. rows): 

vD = -&7#J2Y1 d(x rci ,x rC2 ) 

" " r ci,c 2 
hD = T c ~yi 5Z d(Xric, X r2 c) 

where jjr and )Jc are the number of rows and columns in the grid. 

Figure |4ja) shows the average global diversity observed on the 1000 first genera- 
tions during 100 runs on the QAP. The curves from bottom to top correspond to in- 
creasing values of a from zero to nearly one. Experiments measuring genetic diversity 
show that small migration rate (a close to one) causes islands to dominate others and 
retain global diversity without being able to exchange solutions to produce better re- 
sults. At the opposite, for the cellular model, as a is closed to zero, global diversity 
falls near to zero after 800 generations causing premature convergence and negatively 
affects performances (see figure [3}. Analysis on the QAP show the necessity of main- 
taining diversity to produce new results and the necessity to have enough information 
exchanges between columns. 

Figure |4{b) represents the vertical diversity against the horizontal diversity. The 
contour lines plotted every 100 generations give some information on the speed of de- 
crease of diversity. The more the migration rate decreases (i.e. a increases), the more 




Fig. 4. Global population diversity against generation, with increasing a from bottom 
to top (a) and vertical diversity against horizontal diversity, with increasing a from left 
to right (b). 



the diversity is maintained on each row and subpopulations converge in each column. 
The vertical and horizontal diversities are decreasing with the same speed for the cellu- 
lar model (a — 0) and lower number of interactions between subpopulations helps the 
algorithm to maintain diversity on the rows when a is high. 

Figure [5] shows snapshots of the population diversity during one single run at diffe- 
rent generations. The snapshots are taken from left to right at generations 1, 200, 500, 
1000 and 2000. The parameter a takes values in {0,0.5,0.7,0.86,0.98} from top to 
bottom. Each snapshot shows the genotypic diversity in the neighborhoods of all cells 
on the grid. Color black means maximum diversity and color white means that there 
is no more diversity in the cell's neighborhood. Those snapshots help to understand 
the influence of the anisotropic selection on the genotypic diversity. First, we can see 
that the anisotropic degree influences the dynamic of propagation of good individuals 
on the grid. This propagation is the cause of the loss of diversity in the population. In 
the standard cellular model (a — 0), good individuals propagate roughly circularly. If 
we slightly privilege the vertical direction (a = 0.5) the circles become elliptical. As 
a increases, the dynamic changes and good individuals propagate column by column. 
For extreme values of the anisotropic degree {a close to 1) the migration rate is so low 
that good individuals are stuck in the subpopulations and the sharing of genetic infor- 
mation with other subpopulations is seldom observed. In that case, the selective pres- 
sure is too low and it negatively affects performances. The crossover operator doesn't 
have any effect in the white zones, since they represent cells with no more diversity in 
their neighborhoods. For the standard cellular case, interactions between cells may have 
some effects on performances only at the frontier between the circles. It represents a lit- 
tle proportion of cells on the grid after a thousand generations. For a = 0.86, we can 
see vertical lines of diversity, which means that good individuals appear in each subpop- 
ulations. For example, when we see two adjacent columns colored in grey it means that 
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Fig. 5. Local diversity in the population along generations (left to right) for increasing 
a (top to bottom) 

those columns have been colonized by two different individuals. At generation 2000, a 
good individual has colonized the left of the grid but he still can share information with 
individuals in the grey zones. This means that the migration rate between subpopula- 
tions is strong enough to guarantee the propagation of the genetic information through 
the whole grid. This study showed that the dynamic of the propagation of individuals on 
the grid is strongly related to the anisotropic degree. Once again, it would be interesting 
to see what kind of dynamic appears if we define a local criteria to auto-adapt a during 
a run. This parallel model of GA allows to tune separately the anisotropic degree for 
each cell on the grid and measures during the search process can help to adjust locally 
the selective pressure. 



Conclusion and Perspectives 



This paper presents a unified model of parallel Genetic Algorithms where granularity 
can be continuously tuned from fine grain to coarse grain parallel model. This family 
is based on the new concept of anisotropic selection. We analysed the dynamics of this 
class of pGAs on the well-known QAP problem. We have shown that the anisotropic 
degree plays a major role with regard to the average fitness found. Performances of the 
apGA increases with a until a threshold value (a = 0.86). After this threshold, the 
migration rate between subpopulations in columns may be too small to generate good 
solutions. A study on local diversity shows the interactions between cells for different 
tunings of the apGA. The dynamic of propagation of individuals, which is strongly 
related to the genotypic diversity in the population, is dependent from the anisotropic 
degree of the apGA. Propagation of good individuals is done in circles for low values 
of a and turns to vertical lines for high values of a. Diversity is maintained in the 
population when the anisotropic degree is high, but when it reaches values close to 
the extreme case the few interactions between columns penalize the performances of 
the algorithm. These experimental results lead us to suggest to adjust dynamically the 
migration ratio during a run: by tuning the control parameter a, it would be possible 
to make the algorithm to self-adjust the migration level, depending on global or local 
measures. While theorical and experimental studies on island models are difficult due to 
their complexity, the apGA model could be used as a simple framework for calculations 
on parallel GA. Naturally it would be worth seeing how properties described in this 
paper extend for even more complex problems. 
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